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(54) Method for controlling a system comprising an Information processing apparatus and an 
image sensing device 



(57) An information processing technique allows a 
personal computer to pick up an image through opera- 
tion of a single button (10). When a half-push switch is 
turned on, a CPU activates a window (1 1 1 ) of an image 
pickup application program, causing the window (111) 
to appear in front of the windows (112, 113) of all other 
application programs. An image displayed in the window 
(111) of the image pickup application program is held 
still even if the personal computer is altered in its orien- 
tation. When a full-push switch is turned on, the image 
in the window (1 1 1 ) of the image pickup application pro- 
gram is stored onto an HDD in a suitable format. 
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Description 

[0001] The present invention relates to an information 
processing apparatus, an information processing meth- 
od and a storage medium, capable of permitting images 
to be picked up. 

[0002] In recent years, some personal computers 
have come to embrace multimedia and accommodate 
a CCD camera and its interface, the camera picking up 
images of a user or other objects. Images are picked up 
when an application program addressing such process- 
ing is carried out by the computer. 
[0003] Meanwhile, drastic improvements in the 
processing performance of personal computers have 
made it possible for their operating system to provide as 
a standard feature a multitask environment in which to 
run a plurality of application programs such as a word 
processor applicator a spreadsheet application and an 
image processing application. 

[0004] At the same time, graphically-driven window 
systems have become a standard user interface. This 
has shifted the status of a principal manipulative device 
from the keyboard to the pointing device such as a 
mouse or a track ball. 

[0005] In that operating environment, a user utilizing 
a CCD camera mounted on a personal computer to get 
a picture of an object must first operate a pointing device 
to activate an image pickup application program. The 
user then needs to perform further operations to pick up 
the image. 

[0006] The problem is that it takes time to operate the 
pointing device moving the pointer position, pushing a 
button or manipulating other controls when and where 
appropriate. The time-consuming chore often causes 
the user to miss a perfect moment for a good picture. 
[0007] Respective aspects of the invention are set out 
in claims 1 , 4 ; 7 and 10. 

[0008] A preferred embodiment of the present inven- 
tion provides an apparatus and a method for information 
processing allowing manipulation of a single button to 
operate a personal computer to pick up an image. 
[0009] The invention will now be described by way of 
example with reference to the accompanying drawings, 
throughout which like parts are referred to by like refer- 
ences, and in which: 

Fig. 1 is a schematic view showing a typical struc- 
ture of a personal computer; 
Fig. 2 is a top view of the personal computer; 
Fig. 3 is a side view of the personal computer; 
Fig. 4 is a side view of the personal computer with 
its display part swung open away from its body; 
Fig. 5 is a front view of the personal computer; 
Fig. 6 is a bottom view of the personal computer; 
Fig. 7 is a function block diagram of the personal 
computer; 

Fig. 8 is an explanatory view of a situation where an 
image of an object is picked up by the personal com- 



puter; 

Fig. 9 is a schematic view of a display on an LCD 
in effect before a shutter button is pushed; 
Fig. 1 0 is a schematic view of a display on the LCD 
s in effect when the shutter button is half-pushed; 

Fig. 1 1 is a flowchart of steps constituting a process 
of picking up a still image; and 
Fig. 1 2 is a flowchart of steps constituting a process 
of picking up a motion image. 

[001 0] Figs. 1 through 6 show a typical structure of a 
portable personal computer 1 to which this invention is 
applied. The personal computer 1 is a mini-notebook 
type personal computer that primarily comprises a body 

2 and a display part 3 attached swingingly to the body 
2. Fig. 1 is a perspective view of the computer with the 
display part 3 swung open away from the body 2. Fig. 2 
is a plan view of the computer in Fig. 1 . Fig. 3 is a left- 
hand side view of the computer with the display part 3 
swung shut onto the body 2. Fig. 4 is a right-hand side 
view of the computer with the display part 3 swung open 
1 80 degrees relative to the body 2. Fig. 5 is a front view 
of the computer in Fig. 3. Fig. 6 is a bottom view of the 
computer in Fig. 4. 

[0011] The face of the body 2 comprises a keyboard 
4 and a track point (registered trademark) 5. The key- 
board 4 is used to input characters, symbols, etc., and 
the track point 5 is operated to move a mouse cursor. 
Alsofurnished on the body face is a speaker 8 for sound 
output along with a shutter button 1 0 operated to take a 
picture using the CCD video camera 23 mounted on the 
display part 3. 

[0012] A pawl 1 3 is provided at the upper end of the 
display part 3. As shown in Fig. 3, with the display part 

3 swung closed onto the body 2, the pawl 1 3 hooks onto 
a hole 6 in the body 2. At the front of the body 2 is a slide 
lever 7 furnished in a crosswise movable fashion. The 
slide lever 7 is used to lock and unlock the pawl 1 3 so 
that the pawl 13 is engaged with and disengaged from 
the hole 6. With the pawl 13 unlocked, the display part 
3 may be swung open away from the body 2. Adjacent 
to the pawl 1 3 is a microphone 24 which, as depicted in 
Fig. 6, may pick up sound from both the front and the 
back side of the body 2. 

[0013] The front of the body 2 further comprises a pro- 
grammable power key (PPK) 9. An air outlet 11 is pro- 
vided on the right-hand side of the body 2, as shown in 
Fig. 4. At the lower end in front of the body 2 is an air 
inlet 1 4 as depicted in Fig. 5. To the right of the air outlet 
1 1 is a slot 1 2 that accommodates a PCMCIA (Personal 
Computer Memory Card International Association) card 
(called a PC card). 

[001 4] An LCD (liquid crystal display) 21 for displaying 
images is provided on the front of the display part 3. At 
the upper end of the LCD 21 is an image pickup part 22 
mounted rotatably on the display part 3. More specifi- 
cally, the image pickup part 22 is rotatable to any posi- 
tion within a range of 180 degrees in the same direction 
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as the LCD 21 and in the opposite direction thereof (i. 
e., toward the back). The image pickup part 22 is fur- 
nished with the CCD video camera 23. 
[0015] At the lower end of the display part 3 on the 
body side is a group of lamps including a power lamp 
PL, a battery lamp BL, a message lamp ML and other 
LEDs. Reference numeral 40 in Fig. 3 denotes a power 
switch furnished on the left-hand side of the body 2, and 
reference numeral 25 in Fig. 5 represents an adjusting 
ring used to adjust the focus of the CCD video camera 
23. Reference numeral 26 in Fig. 6 stands for a cover 
that conceals an opening through which to install an ad- 
ditional memory into the body 2, and reference numeral 
41 denotes a hole through which to insert a pin to unlock 
the cover 26. 

[0016] Fig. 7 illustrates an internal structure of the per- 
sonal computer 1 . As shown in Fig. 7, an internal bus 
51 is connected to a CPU (central processing unit) 52, 
a PC card 53 inserted as needed, a RAM (random ac- 
cess memory) 54, and a graphic chip 81. The internal 
bus 51 is coupled to an external bus 55. The external 
bus 55, for its part, is connected to a hard disk drive 
(HDD) 56, an I/O (input/output) controller 57, a keyboard 
controller 58, a track point controller 59, a sound chip 
60, an LCD controller 83, and a modem 50. 
[0017] The CPU 52 is a controller that controls diverse 
computer functions. The PC card 53 is installed as need- 
ed when an optional function is to be added. 
[0018] When the personal computer 1 is booted up, 
an electronic mail program (an application program) 
54A, an auto pilot program (another application pro- 
gram) 54B and the OS (operating program) 54C are 
transferred from the HDD 56 to the RAM 54 and retained 
therein. 

[0019] The electronic mail program 54A is a program 
that exchanges communication messages with an ex- 
ternal entity using a communication line such as a tele- 
phone line and by way of a network. A received mail 
acquisition function is specifically included in the elec- 
tronic mail program 54A. The received mail acquisition 
function checks a mail server 93 to see if a mail box 93A 
therein contains any mail addressed to this program (i. 
e., to the user). If any such mail is found in the mail box 
93A, the received mail acquisition function carries out a 
suitable process to acquire that mail. 
[0020] The auto pilot program 54B is a program that 
starts up and carries out a plurality of predetermined 
processes (or programs) in a predetermined sequence. 
[0021] The OS (operating system) 54C controls basic 
computer functions. Atypical operating system is Win- 
dows 95 (registered trademark). 
[0022] The hard disk drive (HDD) 56 connected to the 
external bus 55 contains the electronic mail program 
56A, auto pilot program 56B, and OS (operating system) 
56C. During the booting process, the OS 56C, auto pilot 
program 56B and electronic mail program 56A are trans- 
ferred successively from the hard disk drive 56 to the 
RAM 54 and stored in the memory. 



[0023] The I/O controller 57 has a microcontroller 61 
equipped with an I/O interface 62. The microcontroller 
61 is constituted by the I/O interface 62, a CPU 63, a 
RAM 64 and a ROM 69 which are interconnected. The 

5 RAM 64 includes a key input status register 65, an LED 
(light-emitting diode) control register 66, a set time reg- 
ister 67, and a register 68. The set time register 67 is 
used to start the operation of a start sequence controller 
76 when a time preset by the user (i.e., starting condi- 

10 tion) is reached. The register 68 holds a correspond- 
ence between a preset combination of operation keys 
(starting condition) on the one hand and an application 
program to be started on the other hand. When the user 
inputs the preset combination of operation keys, the cor- 

15 responding application program (e.g., electronic mail 
program) is started. 

[0024] When the fingertip-operated programmable 
power key (PPK) 9 is pushed, the key input status reg- 
ister 65 gets and retains an operation key flag. The LED 

20 control register 66 is used to control the illumination of 
the message lamp ML indicating that boot-up status of 
an application program (e.g., electronic mail program) 
which is held in the register 68. A desired time of day 
may be set to the set time register 67. 

25 [0025] The microcontroller 61 is connected to a back- 
up battery 74. The battery 74 allows contents of the reg- 
isters 65, 66 and 67 to be retained when power to the 
body 2 is turned off. 

[0026] The ROM 69 in the microcontroller 61 contains 
30 in advance a wake-up program 70, a key input monitor- 
ing program 71, and an LED control program 72. The 
ROM 69 is illustratively composed of an EEPROM (elec- 
trically erasable and programmable read only memory). 
The EEPROM is also called a flash memory. The micro- 
ns controller 61 is connected to an RTC (real-time clock) 
75 that keeps the current time. 

[0027] The wake-up program 70 in the ROM 69 is a 
program that checks to see if a preset time in the set 
time register 67 is reached on the basis of time-of-day 

40 data from the RTC 75. When the preset time is reached, 
the wake-up program 70 starts up a predetermined 
process (or program). The key input monitoring program 
71 continuously monitors whether the PPK 9 is pushed 
by the user. The LED control program 72 controls the 

45 lighting of the message lamp ML. 

[0028] Furthermore, the ROM 69 contains a BIOS 
(basic input/output system) 73. The BIOS is a software 
program that controls exchanges of data (input and out- 
put) between the OS or application software on the one 

50 hand and peripheral devices (e.g., display part, key- 
board, hard disk drive) on the other hand. 
[0029] The keyboard controller 58 connected to the 
external bus 55 controls input from the keyboard 4. The 
track point controller 59 controls input from the track 

55 point 5. 

[0030] The sound chip 60 receives input from the mi- 
crophone 24, and supplies sound signals to a built-in 
speaker 8. 
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[0031] The modem 50 permits connection to a com- 
munication network 92 such as the Internet and to the 
mail server 93 through a public telephone line 90 and 
an Internet service provider 91 . 

[0032] Image data captured by the CCD video camera 
23 are forwarded to a processing part 82 for processing. 
The image data processed by the processing part 82 
are input to the graphic chip 81 connected tothe internal 
bus 51 . The graphic chip 81 stores the input video data 
into an internal VRAM 81 A, and retrieves the data from 
the memory as needed for output to the LCD controller 
83. Given the image data from the graphic chip 81 . the 
LCD controller 83 outputs the data to the LCD 21 for 
display. Back lights 84 are provided to illuminate the 
LCD 21 from the back. 

[0033] The power switch 40 is operated to turn on and 
off the power supply. A half -push switch 85 is activated 
when the shutter button 10 is half-pushed. A full-push 
switch 86 is turned on when the shutter button 1 0 is fully 
pushed. A reverse switch 87 is turned on when the im- 
age pickup part 22 is rotated by 1 80 degrees (i.e., when 
the CCD video camera 23 is rotated into a direction suit- 
able for picking up an image on the opposite side of the 
LCD 21). 

[0034] Fig. 8 illustrates a situation where the personal 
computer 1 is used to pick up an image of an object. The 
user of the personal computer 1 rotates the image pick- 
up part 22 at the upper end of the display part 3 in the 
opposite direction of the LCD 21 so as to take a picture 
of the object in front of the user. 

[0035] Fig. 9 shows a display on the LCD 21 in effect 
before the shutter button 10 is pushed. A window 111 of 
the image pickup application program appears behind 
a window 1 1 2 of a spreadsheet application program and 
a window 1 1 3 of a word processor application program. 
[0036] Fig. 1 0 depicts a display on the LCD 21 in effect 
when the shutter button 1 0 is half-pushed. Half-pushing 
the shutter button 10 activates the window 111 of the 
image pickup application program without clicking on 
the window 111 using a pointer 114. The window 111 
now appears in the foreground in front of the window 
1 1 2 of the spreadsheet application program and the win- 
dow 113 of the word processor application program. An 
image displayed in the window 111 of the image pickup 
application program is held still even if the personal 
computer 1 is altered in its orientation. In this state, fully 
pushing the shutter button 10 stores onto the HDD 56 
the image in the window 111 of the image pickup appli- 
cation program, the storage being made in a suitable 
format. 

[0037] Releasing the half-pushed shutter button 10 
causes the image pickup application program to set free 
the image from its motionless state in the window 111. 
[0038] Fig. 11 is a flowchart of steps constituting a 
process of picking up a still image. In step Sll, the CPU 
52 checks to see if the shutter button 10 is half-pushed 
on the basis of the input from the half-push switch 85. If 
the shutter button 10 is judged to be half-pushed, step 



S12 is reached. In step S12, the CPU 52 activates a 
window of a still image pickup application program, 
causing the window to appear in front of the windows of 
all other application programs. In step S1 3, the CPU 52 
5 holds still the image (captured through the CCD video 
camera 23) displayed in the window of the still image 
pickup application program. 

[0039] In step S14, the CPU 52 checks to see if the 
shutter button 10 is released on the basis of the input 

10 from the half-push switch 85. If the shutter button 10 is 
not judged to be released, step S15 is reached. In step 
S15, the CPU 52 checks to see if the shutter button 10 
is fully pushed on the basis of the input from the full- 
push switch 86. If the shutter button 10 is judged to be 

15 fully pushed, step S16 is reached. In step S16, the im- 
age in the window of the still image pickup application 
program is stored onto the HDD 56 in an appropriate 
format. This terminates the process of still image pickup. 
[0040] If the shutter button 1 0 is not judged to be fully 

20 pushed in step S1 5, step S1 4 is reached again. The op- 
erative state of the shutter button 10 is then continuously 
monitored for judgment. 

[0041] If the shutter button 1 0 is not judged to be half- 
pushed in step S11 or if the shutter button 10 is judged 
25 to be released in step S14, then the still image pickup 
process is terminated. 

[0042] As described above, the user is able to pick up 
a still image of the object by operating the shutter button 
10 alone. 

30 [0043] A process of picking up a motion image will 
now be described. Fig. 1 2 is a flowchart of steps consti- 
tuting the motion image pickup process. In step S21 , the 
CPU 52 checks to see if the shutter button 10 is half- 
pushed on the basis of the input from the half-push 

35 switch 85. If the shutter button 10 is judged to be half- 
pushed, step S22 is reached. In step S22, a window of 
a motion image pickup application program is activated 
and made to appear in front of the windows of all other 
application programs. 

40 [0044] In step S23, the CPU 52 checks to see if the 
shutter button 10 is released on the basis of the input 
from the half-push switch 85. If the shutter button 10 is 
not judged to be released, step S24 is reached. In step 
S24, the CPU 52 checks to see if the shutter button 10 

45 is fully pushed on the basis of the input from the full- 
push switch 86. If the shutter button 10 is judged to be 
fully pushed, step S25 is reached. In step S25, the CPU 
52 checks to see if a motion image is being picked up. 
If it is judged that no motion image is being picked up in 

50 step S25, step S26 is reached. In step S26, a motion 
image starts to be picked up, and the process is termi- 
nated. If a motion image pickup is judged to be in 
progress in step S25, step S27 is reached in which the 
motion image pickup is stopped and the process is ter- 

55 minated. 

[0045] If the shutter button 1 0 is not judged to be fully 
pushed in step S24, step S23 is reached again. The op- 
erative state of the shutter button 10 is then continuously 
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monitored for judgment. 
[0046] If the shutter button 1 0 is not judged to be half- 
pushed in step S21 or if the shutter button 10 is judged 
to be released in step S23, then the motion image pickup 
process is terminated. 

[0047] As described, the user is able to carry out or 
stop the motion image pickup by operating the shutter 
button 1 0 alone. 

[0048] Because the shutter button 1 0 alone needs to 
be operated for picking up still and motion images, the 
user will not likely miss a perfect moment for a good pic- 
ture. 

[0049] In the description above, the still image pickup 
application program or the motion image pickup appli- 
cation program was assumed to be already running. Al- 
ternatively, any one of these programs may be booted 
and executed the moment the shutter button 10 is half- 
pushed. 

[0050] Computer programs designed to perform the 
above-described processes may be retained on such 
storage media as magnetic disks, CD-ROMs, or solid 
state memories when offered to users. The programs 
may also be distributed by use of such communication 
media as networks and satellites. 
[0051] Through the use of the inventive information 
processing apparatus, information processing method 
and storage medium, the image pickup application pro- 
gram is activated when an image of an object is picked 
up and the user's suitable operations are detected. Only 
the shutter button needs to be operated for picking up 
still and motion images. 

[0052] As many apparently different embodiments of 
this invention may be made without departing from the 
scope thereof, it is to be understood that the invention 
is not limited to the specific embodiments thereof except 
as defined in the appended claims. 



Claims 

1. An information processing apparatus which has a 
graphic user interface adopting windows and which 
is capable of selecting and operating any of a plu- 
rality of application programs, said information 
processing apparatus comprising: 

image pickup means for picking up an image of 
an object; 

detecting means for detecting operations per- 
formed by a user; and 

controlling means for controlling an application 
program so that said application program is ac- 
tivated to pick up said image when said detect- 
ing means detects operations. 

2. An information processing apparatus according to 
claim 1 , wherein said detecting means detects a 
first and a second operation performed by said user; 



8 
and 

wherein said controlling means activates a 
window for said application program for picking up 
said image if said detecting means detects said first 
s operation, said controlling means further causing 
said image pickup means to pick up said image 
when said detecting means detects said second op- 
eration. 

10 3. An information processing apparatus according to 
claim 2, wherein said first operation is a half-push 
button operation and said second operation is a full- 
push button operation. 

15 4. An information processing method for use with a 
graphic user interface adopting windows and capa- 
ble of selecting and operating any of a plurality of 
application programs, said information processing 
method comprising the steps of: 

20 

picking up an image of an object; 
detecting operations performed by a user; and 
controlling an application program so that said 
application program is activated to pick up said 
25 image when operations are detected in said de- 

tecting step. 

5. An information processing method according to 
claim 4, wherein said detecting step detects a first 

30 and a second operation performed by said user; 
and 

wherein said controlling step activates a win- 
dow for said application program for picking up said 
image if said detecting step detects said first oper- 
as ation, said controlling step further causing said im- 
age pickup step to pick up said image when said 
detecting step detects said second operation. 

6. An information processing method according to 
40 claim 5 ; wherein said first operation is a half-push 

button operation and said second operation is a full- 
push button operation. 

7. A storage medium for storing a program in a manner 
45 readable by an information processing apparatus 

which has a graphic user interface adopting win- 
dows and which is capable of selecting and operat- 
ing any of a plurality of application programs, said 
program allowing said information processing ap- 
50 paratus to execute an information processing meth- 
od comprising the steps of: 

picking up an image of an object; 
detecting operations performed by a user; and 
55 controlling an application program so that said 

application program is activated to pick up said 
image when operations are detected in said de- 
tecting step. 
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A storage medium according to claim 7, wherein 
said detecting step detects a first and a second op- 
eration performed by said user; and 

wherein said controlling step activates a win- 
dow for said application program for picking up said 
image if said detecting step detects said first oper- 
ation, said controlling step further causing said im- 
age pickup step to pick up said image when said 
detecting step detects said second operation. 

A storage medium according to claim 8, wherein 
said first operation is a half -push button operation 
and said second operation is a full-push button op- 
eration. 
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10. A program for an information processing apparatus 
which has a graphic user interface adopting win- 
dows and which is capable of selecting and operat- 
ing any of a plurality of application programs, said 
program allowing said information processing ap- 20 
paratus to execute an information processing meth- 
od comprising the steps of: 

picking up an image of an object; 
detecting operations performed by a user; and 25 
controlling an application program so that said 
application program is activated to pick up said 
image when operations are detected in said de- 
tecting step. 

30 



35 



40 



45 



50 



55 



6 



EP 0 969 400 A2 




7 



EP 0 969 400 A2 



F I G. 2 



21 



□ 



13 24 



la 




ML BL PL 



I; I 



4 



a 

a 



^3-^10 



nm 



ZLTTZmDLTTDLXrn 


znauzL 


ODTTTTI1 




rn i 


znin 


nnmnLiq 

! o — »=: 


I 


nnr 



8 



EP 0 969 400 A2 




9 



EP 0 969 400 A2 




10 



EP 0 969 400 A2 




11 



EP 0 969 400 A2 



J_ 



—ten 
<r) 
zoo 
en 



in 

ZD 
CQ 

— I CO 



I 



I 



; o at: i o 

— OiZj i 

a=a- o: o 

J — 1 — ir-r- 1 



-el 



< m o 
^- 



CD 

m 

s 



■Si— - 
, 1 i n. + 

i OO i 



o CD CD CD 



m 

) 

a 
o 





12 



EP 0 969 400 A2 




13 



EP 0 969 400 A2 



ro 



CD 



<3 



□ 
□ 
□ 



COO 

COCL 
OCLi 1 




CM 



D 




CO 



O 
O 



v 



CO 



14 



EP 0 969 400 A2 



ro 



o 



□ 
□ 



LU 



□ 
□ 
□ 







-J 




LU 








>- 




< 




1 


LUO 


o_ 




CO 




Q 






m2 




O 



CM 




□ 



LU 



CO 



CD 



O 
O 



8 

O 



o 

8 



cr 
en 



15 



EP 0 969 400 A2 



FIG. 11 
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